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Abstract 

Lipschitz extensions were recently proposed as a tool for designing node differentially pri¬ 
vate algorithms. However, efficiently computable Lipschitz extensions were known only for 
1-dimensional functions (that is, functions that output a single real value). In this paper, we 
study efficiently computable Lipschitz extensions for multi-dimensional (that is, vector-valued) 
functions on graphs. We show that, unlike for 1-dimensional functions, Lipschitz extensions 
of higher-dimensional functions on graphs do not always exist, even with a non-unit stretch. 
We design Lipschitz extensions with small stretch for the sorted degree list and for the degree 
distribution of a graph. Crucially, our extensions are efficiently computable. 

We also develop new tools for employing Lipschitz extensions in the design of differentially 
private algorithms. Specifically, we generalize the exponential mechanism, a widely used tool 
in data privacy. The exponential mechanism is given a collection of score functions that map 
datasets to real values. It attempts to return the name of the function with nearly minimum 
value on the data set. Our generalized exponential mechanism provides better accuracy when 
the sensitivity of an optimal score function is much smaller than the maximum sensitivity of 
score functions. 

We use our Lipschitz extension and the generalized exponential mechanism to design a node- 
differentially private algorithm for releasing an approximation to the degree distribution of a 
graph. Our algorithm is much more accurate than algorithms from previous work. 
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1 Introduction 


The area of differential privaey studies how to output global information contained in a database 
while protecting privacy of individuals whose information it contains. Typically, the datasets 
considered are tabular databases, containing one row of information per person. While the area 
came a long way in the last decade in terms of the richness of information that can be released 
with differential privacy for tabular databases, we are lagging behind in our understanding of graph 
datasets that also contain relationships between various participants. Such datasets are used, for 
example, to capture relationships between people in a social network, communication patterns, and 
romantic relationships. 

There are two natural variants of differential privacy that are suited for graph datasets: edge 
differential privacy and node differential privacy. Intuitively, the former protects relationships 
among individuals, while the latter protects each individual, together with all his/her relationships. 
Edge privacy is a weaker notion and has been studied more extensively, with algorithms now known 
for the release of subgraph counts and related scalar-valued functions [Ml ESI Eg ESI [251 HH, the 
degree distribution [i2iEg[iai2ii[i6], cut densities mm and the parameters of generative graph 
models [MUHlEgEgEZI. Node differential privacy is a much stronger privacy guarantee, but is 
much harder to attain because it guards against larger changes in the input. Until recently, there 
were no known differentially private algorithms that gave accurate answers on sparse graphs, even 
for extremely simple statistics. In 2013, Blocki et al. [1[, Kasiviswanathan et al. [SO], Chen and 
Zhou [g proposed two new techniques for node private algorithms: (i) using projections whose 
smooth sensitivity could be bounded (combined with mechanisms that add noise tailored to the 
smooth sensitivity m), and (ii) using Lipschitz extensions (combined with the standard Laplace 
mechanism). The latter technique yielded much more accurate algorithms than the former. In 
particular, it was used to obtain accurate node differentially private algorithms for computing 
subgraph counts and related statistics. 

However, efficiently computable Lipschitz extensions were known only for 1-dimensional func¬ 
tions (that is, functions that output a single real value). In this paper, we study efficiently com¬ 
putable Lipschitz extensions for multi-dimensional (that is, vector-valued) functions. We show that, 
unlike for 1-dimensional functions, Lipschitz extensions of higher-dimensional functions do not al¬ 
ways exist, even with a non-unit stretch. We design Lipschitz extensions with small stretch for the 
sorted degree list and for the degree distribution of a graph. Our extensions can be computed in 
polynomial time. 

We also develop new tools for employing Lipschitz extensions in the design of differentially 
private algorithms. Specifically, we generalize the exponential mechanism of McSherry and Talwar 
|31j , a widely used tool in data privacy. Our generalized mechanism provides better accuracy when 
the sensitivity of an optimal score function is much smaller than the maximum sensitivity of score 
functions. 

We use our Lipschitz extension and the generalized exponential mechanism to design a node 
differentially private algorithm for releasing an approximation to the degree distribution of a graph. 
Our algorithm is much more accurate than those from previous work [llEo]- 

Lipschitz extensions. Lipschitz extensions are basic mathematical objects studied in functional 
analysis. 
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Definition 1.1 (Lipschitz constant). Let f : X ^ Y be a function from a domain X to a range Y 
with associated distance measures dx and dy ■ Function f has Lipschitz constant c (equivalently, is 
c-Lipschitz) if dy^f^x), f{x')) < c ■ dx{x,x') for allx,x' £X. 

Definition 1.2 (Lipschitz extension). Consider a domain X and a range Y with associated distance 
measures dx and dy, and let X' C X. Fix constants c > 0 and s > 1. Given a c-Lipschitz function 
f'-.X'^Y, a function f : X ^ Y is a Lipschitz extension of f' from X' to X with stretch s if 

1. f is an extension of f, that is, f{x) = f'{x) on all x ^ X' and 

2. f is s ■ c-Lipschitz. 

If s = 1, then we call f a Lipschitz extension of /' from X' to X (omitting the stretch). 

Functional analysts have devoted considerable attention to determining, for given metric spaces 
X, X' and Y, whether Lipschitz extensions with stretch 1 exist for all functions f : X ^ Y. In 
contrast to this paper, the focus is mostly on continuous function spaces. 

Lipschitz extensions of real-valued 1-dimensional functions with stretch 1 always exist [2^. We 
show that it is not true, in general, for multi-dimensional functions on graphs, even with non-unit 
stretch. The technical core of this paper is the construction of an efficiently computable extension 
of the degree distribution, a high-dimensional function on graphs, with small stretch. 

Metrics on Graphs. Let G denote the set of all finite labeled, unweighted undirected graphs. 
When the input data set is a graph in G, there are two natural notions of “neighbor” (or adjacency). 
Two graphs G and G' are edge neighbors if they differ in one edge. Two graphs G and G' are node 
neighbors if one can be obtained from the other by removing one node and its adjacent edges. These 
two notions of neighbor induce two metrics on G, node distance (dnode) and edge distance (dedge)- 

Why are Lipschitz Extensions Useful for Privacy? A randomized algorithm A is node 
differentially private if, for any two datasets that are “neighbors” in an appropriate sense, the 
distributions on the algorithms outputs are close in a multiplicative sense. Notions of stability and 
sensitivity play a key role in the design of differentially private algorithms. Differential privacy 
itself can be seen as a stability requirement, since the algorithm must map neighboring graphs to 
nearby distributions on outputs. 

The two basic building blocks for designing differentially private algorithms, the Laplace and 
exponential mechanisms, rely on the global sensitivity of a function /, which is the Lipschitz constant 
of / viewed as a map from data sets (e.g., G equipped with dnode) to {i.e., equipped with £i). 
The Laplace mechanism [lOj shows that one can satisfy differential privacy by releasing f{G) with 
additive noise proportional to the node global sensitivity in each coordinate. 

The difficulty with employing the Laplace mechanism directly is that many useful functions on 
graphs are highly sensitive to the insertion or removal of a well-connected vertex. For example, 
the number of connected components of a graph may go from n to 1 with the insertion of a single 
vertex. The degree distribution of a graph can also change drastically, shifting up by 1 in every 
coordinate (as one vertex can increase the degree of all other vertices). This difficulty generally 
remains even if we shift from global sensitivity to more local notions (as in |34] 1 (roughly, interesting 
graphs such as those with low average degree are “near” other graphs with a vastly different value 
for the function). 
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One can get around this by focusing on a “nice” or “typical” subset of the space Q where the 
function / has low global sensitivity [H 120116] . For example, let be the set D-bounded graphs, 
that is, graphs of maximum degree at most D. Many functions have bounded sensitivity (Lipschitz 
constant) on . The number of triangles in a graph, for instance, changes by at most (^) among 
node-neighboring graphs of degree at most D, and the degree list changes by at most 2D in ii. 

Given a function / that has low Lipschitz constant on “nice” graphs, if we find an efficiently 
computable Lipschitz extension / that is defined on all of then we can use the Laplace mechanism 
to release f{G) with relatively small additive noise. The lower the stretch of the extension, the 
lower the overall noise. The result will be accurate when the input indeed falls into, or near, the 
class of “nice” graphs. Interestingly, the class of “nice” graphs need not contain the input for the 
answer to be accurate—in our main application, we use as the set of “nice” graphs, but D is 
set much lower than the actual maximum degree of the input. 

Existence and efficiency of Lipschitz extensions. Motivated by this methodology, we ask: 
when do Lipschitz extensions exist, and when do they admit efficient algorithms? The existence 
question has drawn interest from functional analysis and combinatorics for nearly a century |29L 
EH [36l EH ESI EH U El E2l 1331 E31 ES]; see Lee and Naor [23] for an overview. For any real¬ 
valued function / : —?■ M, there exists an extension / : ^ M whose node sensitivity is the 

same as that of /. Kasiviswanathan et al. [20!) Chen and Zhou [Oj constructed polynomial-time 
computable Lipschitz extensions from to Q of several real-valued functions on graphs. The 
techniques in [IIEOIEI apply to functions that count structures in a graph, possibly with weights 
(for example, the number of edges in a graph, the number of triangles in a simple graph; in a graph 
where vertices and edges have attributes, one could count edges that link nodes labeled by different 
genders in a social network, or triangles involving vertices labeled with different scientific fields in 
a collaboration graph). 

Prior work on constructions of higher-dimensional extensions focused on extending functions 
on a metric space X, where X is given explicitly as input (say, as a distance matrix) |23LI26j . Such 
constructions can, at best, run in time polynomial in the size of X. The size of is infinite, 
and even restricting to graphs on at most n vertices leaves a set that is exponentially large in 
n. Moreover, generic constructions have stretch at least polynomial in the log of the metric’s 
cardinality, at least ^/n in our case. 

1.1 Our Contributions 

In this paper, we demonstrate that efficient and nontrivial constructions of Lipschitz extensions for 
high-dimensional graph summaries are possible. We also develop new machinery for using these 
extensions in the context of differentially private algorithms. 

Lipschitz Extension of the Degree List (Section]^. Our main technical contribution is a 
polynomial-time, constant-stretch Lipschitz extension of the sorted degree list, viewed as a function 
from to to all of Q. Here denotes the li metric on the space of finite-length real sequences, 
where sequences of different length are padded with O’s to compute distance. 

Given an arbitrary graph G, our function /d(G) outputs a nonincreasing real sequence of length 
IVgI- If the maximum degree of G is D or less, the output is the sorted list of degrees in G. The 
output can be thought of as a list of “fractional degrees”, where “fractional edges” are real weights 
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in [0,1] and the “fractional degree” of a vertex is the sum of the weights of its adjacent edges. 
The weights are selected by minimizing a quadratic function over the polytope of s-t flows in a 
directed graph closely related to G. Previous work [20] had shown that the value of the maximum 
flow in the graph has low sensitivity; by introducing the quadratic penalty, we give a way to select 
an optimal flow that changes slowly as the graph itself changes. Introducing a strongly convex 
penalty (or regularizer) to make the solution of an optimization problem stable to changes in the 
loss function is common in machine learning. In our setting, however, it is the constraints of the 
convex program that change with data, and not the loss function. 

Theorem 1.3. There is a Lipschitz extension of deg-list, viewed as a funetion taking values in 
from to Q with stretch 3/2 that can he computed in polynomial time. 

The sorted degree list has ii sensitivity D on The extension foiG) has £i sensitivity at 
most 3D (the stretch is thus at most 3/2). Previous results on Lipschitz extensions only imply the 
existence of an extension with stretch at least n; see Section]^ for discussion of the general results. 

We use our Lipschitz extension of the sorted degree list to get a Lipschitz extension of the degree 
distribution (a list of counts of nodes of each degree) and the degree CDF (a list of counts of nodes 
of at least each given degree). These functions condense the information to a D-dimensional vector 
(regardless of the size of the graph), making it easier to release with node-differential privacy. 

Generalized Exponential Mechanism for Scores of Varying Sensitivity (Section [^. 

One of the difficulties that arises in using Lipschitz extensions for differentially private algorithms is 
selecting a good class of inputs from which to extend. For example, to apply our degree distribution 
extension, we must select the degree bound D. More generally, we are given a collection of possible 
extensions fi,fk, each of which agrees with / on a different set and has different sensitivity Aj. 

For a large class of extensions, we can abstract the task we are faced with as a private opti¬ 
mization problem: given a set of real-valued functions qi, ■.■,qk, the goal is to output the index i 
of a function with approximately minimal value on the data set x (so that q/x) ~ min* q/x)). (In 
our setting, the qi functions are related to the error of the approximation fi on x). Suppose that 
each qi has a known (upper bound on) global sensitivity Aj. The error of an output i on input x 
is the difference q-fx) — min* qi{x). 

The exponential mechanism of McSherry and Talwar m, a widely used tool in differentially 
private algorithms, achieves error that scales with the largest of the sensitivities. Specifically, for 
every /3 > 0, with probability 1 — /3, the output i satisfies q/x) < min* qi{x) Amax ■ ^ .^j^ere 
Araax ~ max^ A^. 

In contrast, we give an algorithm whose accuracy scales with the sensitivity of the optimal score 
function Aj* where i* = argmin^ q/x). Our mechanism requires as input an upper bound /3 > 0 on 
the desired probability of a “bad” outcome; the algorithm’s error guarantee depends on this (3. 

Theorem 1.4 (Informal). For all settings of the input parameters / G (0,1), e > 0, the Generalized 
Exponential Mechanism is e-differentially private. For all inputs x, the output i satisfies 

q/x) < min (^q/x) -\- Aj • _ 

This guarantee can be much tighter than that of the usual exponential mechanism. For instance, 
in our setting, the Aj’s grow exponentially with i yet on sparse graphs, the best choice of Aj is 
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for i relatively small. (Also, the issue is not merely with the error guarantee. The exponential 
mechanism provides bad outputs for many inputs where the true minimizer has low sensitivity.) 

We can use our algorithm for selecting the sensitivity parameter for the Lipschitz extensions 
of graph functions in HEolE] and in this work. (These parameters are sometimes interpretable 
as a degree bound, as in the case of the degree distribution, but not always; for example, when 
computing the number of triangles, the parameter is a bound on the number of triangles involving 
any one vertex). This allows the algorithm to adapt to the specific input. The guarantee we get is 
that the error of the overall algorithm (that is approximating some function of an n-node graph) is 
at most O(loglogn) times higher than one would get with the best Lipschitz constant. In contrast, 
the parameter selection method of Chen and Zhou [6] provides only a O(logn) guarantee on the 
error blow-up, and is specific to the extensions they construct. 

Differentially Private Algorithms for Releasing the Degree Distribntion (Section]^. 

We can combine the Lipschitz extension of the degree list and the parameter selection algorithm 
to get a differentially private mechanism for releasing the degree distribution of a graph that 
automatically adapts to the structure of the graph. 

We show that our algorithm provides an accurate estimate on a large class of graphs, including 
graphs with low average degree whose degree distribution is heavy-tailed. We measure accuracy 
in the ii norm, normalized by the number of nodes in the graph — i.e., we deem the algorithm 
accurate if the total variation distance between the true degree distribution and the estimate is 
small. 

This measure goes to 0 for graphs of low average degree in which the tail of the degree distri¬ 
bution decreases slightly more quickly than what trivially holds for all graphs. If d is the average 
degree in a graph, Markov’s inequality implies that the fraction of nodes with degree above t ■ d is 
at most 1/t. We assume that this fraction goes down as l/t“ for a constant a > 1. The condition 
is called a-decay. Our algorithm need not be given a or the average degree of the graph; these 
are implicitly taken into account by parameter selection. Our assumption is satisfied by all the 
well-studied social network models we know of, including so-called scale-free graphs [7]. 

2 Definitions 

Notation. We use [n] to denote the set {1,... ,n}. For a graph, (V,E), d{G) = 2|i?|/|l/| is the 
average degree of the graph G and deg^(G) denotes the degree of node u G F in G. When the 
graph referenced is clear, we drop G in the notation. The asymptotic notation On(')) On(') is defined 
with respect to growing n. Other parameters are assumed to be functions independent of n unless 
specihed otherwise. 

2.1 Graphs Metrics and Differential Privacy 

Definition 2.1 ((e, (5)-edge/node-privacy). A randomized algorithm A is (e, 5)-edge-private (re¬ 
spectively, node-privatej if for all events S in the output space of A, and edge (respectively, node) 
neighbors Gi,G 2 , 

Pr[A(Gi) G 5] < exp(e) x Pr[A(G 2 ) G 5] + <5. 

When (1 = 0, the algorithm is e-edge-private (respectively, e-node-private). In this paper, if node or 
edge privacy is not specified, we mean node privacy by default. 
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For simplicity of presentation, we assume that n = |y|, the number of nodes of the input graph 
G, is publicly known. This assumption is justified since, as we will see, one can get an accurate 
estimate of |y| by running a node-private algorithm. 

Both variants of differential privacy “compose” well, in the sense that privacy is preserved (albeit 
with slowly degrading parameters) even when the adversary gets to see the outcome of multiple 
differentially private algorithms run on the same data set. 

Lemma 2.2 (Composition, post-processing j3Ul [5]i. If an algorithm A runst randomized algorithms 
^ 1 ,... ,At, each of which is {e, 6)-differentially private, and applies a randomized algorithm g to 
the outputs, i.e., A{G) = g{Ai{G),... ,At{G)), then A is {te,t6)-differentially private. 


2.2 Basic Tools 


Global Sensitivity and the Laplace Mechanism. In the most basic framework for achieving 
differential privacy, Laplace noise is scaled according to the global sensitivity of the desired statistic 
/. This technique extends directly to graphs as long as we measure sensitivity with respect to the 
metric used in the definition of the corresponding variant of differential privacy. Below, we explain 
this (standard) framework in terms of node privacy. Let G denote the set of all graphs. 


Definition 2.3 (Global Sensitivity [lOjl. The G-global node sensitivity of a function f : G ^ W 
is: 


A/= inax ||/(G'i)-/(G 2 )||i. 

Gi,G2 node neighbors 

Equivalently, Af is the Lipschitz constant of a function viewed as a map from {G,d^ode) to 


For example, the number of edges in an n-node graph has node sensitivity n, since adding or 
deleting a node and its adjacent edges can add or remove at most n edges. In contrast, the number 
of nodes in a graph has node sensitivity 1. 

A Laplace random variable with mean 0 and standard deviation y/2X has density h{z) = 
(l/(2A))e“l^l/^. We denote it by Lap(A). 

Theorem 2.4 (Laplace Mechanism [TO]). The algorithm A{G) = f{G) -|-Lap(A//e)^ (which adds 
i.i.d. noise Lap(A//e) to each entry of f{G)) is e-node-private. 


Thus, we can release the number of nodes |B| in a graph with noise of expected magnitude 1/e 
while satisfying node differential privacy. Given a public bound n on the number of nodes, we can 
release the number of edges \E\ with additive noise of expected magnitude n/e. 


Exponential Mechanism. Suppose data sets are members of a universe U equipped with a 
neighbor relation (for example, U = G with vertex neighbors). Suppose we are given a collection of 
functions qi, ...,qk, from 1/ to M such that for each i G [k], the function gj(-) has sensitivity at most 
A. The exponential mechanism (McSherry and Talwar [3T]) takes a data set and aims to output 
an index i for which qfG) has nearly minimal value at G, that is, such that qfG) ~ minjg'j(G). 
The algorithm A samples an index i such that Pr(Al(G) = i) oc exp {^qiiG)) . 

Lemma 2.5 (Exponential Mechanism [HI]). The algorithm A is e-differentially private. Moreover, 
with probability at least 1 — rj, its output i satisfies q-fG) < min* {qi{G)) _ 

There is a simple, efficient implementation of the exponential mechanism that adds exponential 
noise to each score function and reports the maximizer of the noisy scores (see, e.g., P Sec. 3.4]). 
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3 General Results on Lipschitz Extensions 

A number of basic results from functional analysis apply to our setting. Let £p denote the set 
equipped with the ip metric. 

When y = M (with the usual metric), a Lipschitz extension always exists |29) . The classic 
construction, given a c-Lipschitz function / : X —>• M, defines / : X' —)• M as 

fiy)= inf (/(x)+c-■ 

x£X 

The function / is also c-Lipschitz, but need not necessarily be easy to compute even if / admits 
efficient algorithms. 

Blocki et al. jl], Kasiviswanathan et al. [20], Chen and Zhou |6] constructed polynomial-time 
Lipschitz extensions from to Q of several real-valued functions on graphs (see Introduction). 

In this work, our focus is on higher-dimensional functions on graphs, i.e., functions that map 
graphs into for p > 1. As with one-dimensional functions, there always exist stretch-1 extensions 
of functions that take values in i^ for any dimension p, since one can separately find an extension 
for each coordinate of /. It is also true for if, since if is isomorphic to if^. However, stretch-1 
extensions need not exist when Y = i^ or if for larger p. There is a growing body of theory on the 
minimal stretch required for extensions among different spaces; see [21123] for a concise summary 
of known general results on the problem. 

Our first result is that one cannot always get stretch-1 extensions for functions from to if 
or if. We prove it at the end of this section. It is the only lower bound on extendability for these 
metrics we are aware of. 

Proposition 3.1. Consider the vertex distanee on Q. There is an absolute constant c > 1 such 
that: (1) for all p > 3, there exist symmetric functions from to if that do not admit a stretch-c 
extension to Q; (2) for all p>2, there exist symmetric functions from to if that do not admit 
a stretch-c extension to Q. 

This lower bound extends to edge distance on Q (we omit the proof). Moreover, for edge 
distance, it is essentially tight: a result of Blocki et al. [^ on smooth projections implies that every 
function on which is Lipschitz under the edge distance metric on can be extended to all 
of G with stretch at most 3, regardless of the output metric. However, the construction does not 
apply to vertex distance on graphs. 

For the vertex distance on G^, known results yield extensions with stretch that is polynomial 
in either p or re (the size of the graph). We outline these briefly: Lee and Naor |23l Theorem 
1.6] show that one can get extensions with stretch 0{p{X)), where p{X) is the doubling dimension 
of the metric space X (in our case, G^ or Gn). Unfortunately, the vertex metric on Gn 
doubling dimension at least re, even for D = A and even if we identify isomorphic graphs (see 
Appendix for formal definitions and a proof). Makarychev and Makarychev [26] show that 
functions from any metric on N points can be extended to an arbitrary containing space with 
stretch at most O (log X/log log X). Since logX is approximately nD for Gn, this again yields 
large stretch. Finally, another general approach, based on the dimension of the image space, yields 
stretch p and ^/p for maps into if and if respectively (in our case, one can obtain this by separately 
extending each of the p coordinates of the output). 
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Proof of Proposition 3.1 . Our proof is inspired by the example of Benyamini and Lindenstrauss [2] 
of spaces X ’f-X' and a function / : X —?■ such that there is no stretch-1 extension of / from X 
to X'. 

We start with the case of maps into i\. Let X' denote the metric space {a,b,c,d,e} with all 
pairwise distances among X = {a, b, c, d} equal to 2, and distances dx'ix, e) = 1 for x G X (pictured 
as a graph below). Consider the function / ; X —)• that maps X to the corners of a particular 
tetrahedron: 



( 1 ) 


The function / is 2-Lipschitz if we view the image as but there is no way to extend it to all of 
X' in either metric without stretching the Lipschitz constant. To satisfy the Lipschitz constraint 
/(e) has to be exactly halfway between every pair in the set {/(a), /(&), /(c), f{d)} (since it has to 
be at distance at most 2 from each of the points). The points that are halfway from a to b have 
third coordinate 1; the points halfway from c to d have third coordinate -1; there is no intersection 
between the two sets, and hence no possible value for /(e). Any value for /(e) results in a stretch 
of at least some absolute constant c > 1. 

We can lift this example to other domains X C X'. For example, we can take X' to be and 
let a = (1,0,0,0), b = (0,1, 0,0), c = (0, 0,1, 0), d = ((0, 0, 0,1) and e = (0,0,0,0). 

Lifting the example to C ^ is a bit messier. Fix d at least 4. Let Gq be a graph on at 
least 4(d — 2) vertices with maximum degree at most d — 1 and no nontrivial automorphisms (a 
sufficiently large random graph satisfies the criteria with high probability O Chap. 9]). We create 
a larger graph H by adding four vertices {t,u,v,w} to Go, among which all possible edges exist, 
and such that t, u, v and w are connected to a disjoint subsets of d — 2 vertices in Gq (this is possible 
since Gq must have at least 4(d — 2) vertices). The vertices t, u, v, w have degree d -|- 1 in H. 



To embed our counterexample in G, let e = H, and let {a, b, c, d} be the graphs obtained by deleting 
one of t,u,v,w (respectively) from H. The four graphs a,b,c,d lie in G^, and no pair of them is 
isomorphic (since u,v,w, are connected to disjoint sets of a graph with no automorphisms). The 
vertex distance between any pair of graphs in a, b, c, d is 2, and their distance from e is 1. We can 
set the values of / on a, b, c, d as in Q (this is consistent with the requirement that / be symmetric 
since the graphs are not isomorphic). By the reasoning above, / is 2-Lipschitz but there is no way 
to assign a value to /(e) without increasing the stretch of /. 

We must still show that it is possible to assign values to functions on the remaining graphs in 
G^ without increasing the Lipschitz constant. 







For the graphs G that can be obtained by exactly two of t, u, v, w (along with corresponding 
edges) to Go, there two of the graphs in {a, b, c, d} that are at distance 1 from G. We set /(G) to 
be the average of the values of / at these two nearest graphs (for example, G + {t, u} is at distance 
1 from graphs c and d] we set /(G + {t, n}) = (0, 0, —1). Note that /(G + {t, n}) is at distance 2 
from /(c) and f{d), as required. For all other graphs, G G Q^, we set /(G) = (0,0,0). One can 
verify by inspection that the 2-Lipschitz property is satisfied on all of by /. 

Finally, we note that an even simpler example works for maps into £ 2 - Starting with the same 
spaces X and X', we can consider a function / : X —)• that maps {a,b,c} to the corners of an 
equilateral triangle with side-length 1. The map is ^ -Lipschitz on {a, b, c}, but cannot be extended 
to all of X' (since there is no point at distance | of all three corners. Lifting the example to G^ C G 
is similar to the £i case. □ 

4 Lipschitz Extensions of the Degree List and Distribution 

4.1 Lipschitz Extension of the Degree List 

In this section, we give a Lipschitz extension of the degree list. For an re-node graph G, let 

deg-list(G) = sort{degi{G),...,deg\v(G)\{G)) 

denote the list of degrees of G sorted in nonincreasing order. 

We view the degree list as an element of M* (the set of finite sequences of real numbers). We 
equip the space with the £i distance, where the sequences of different lengths are padded with O’s 
to allow comparison. This representation is convenient for handling node additions and deletions. 

The global £i sensitivity (under node insertion and removal) of the degree list on D-bounded 
graphs is 2D because the unsorted degree list has sensitivity 2D and, as Hay et al. m observed, 
sorting does not increase the £i distance between vectors. We construct an extension that agrees 
with deg-list on G^ and has global sensitivity at most 3D. 

Before explaining our construction, we consider a simpler “straw man” construction to illustrate 
the problem’s difficulty: suppose that given the degree list deg-list(G), we obtain /z)(G) by rounding 
all degrees above D down to D. This will not affect the degrees in graph with maximum degree 
D, but it is not 0{D) Lipschitz: consider a star graph on re vertices, with one vertex of degree 
re — 1 and re — 1 vertices of degree 1. Simple rounding would report /(G) as (D, 1,...., 1). But 
the graph has a neighbor G' with no edges at all, for which the reported degree list would be all 
O’s. Those vectors differ by re -|- D — lin the £i norm. One can try simple ways of dropping very 
high-degree vertices (an idea called “projection” in [11|20]), but those do not yield uniform bounds 
on the sensitivity of the resulting degree sequence and result in more noise being added for privacy. 

Like in |20], our starting point is the construction of the flow graph G' for graph G. Ka- 
siviswanathan et al. [20] proved that the value of the maximum flow in G' is a Lipschitz extension 
of the number of edges in G. We will use the flow values on certain edges as a proxy for degrees 
of related vertices. The main challenge is that, whereas the value of the maximum flow in G' is 
unique, the actual flow on specific edges is not. 

Definition 4.1 (Flow graph). Given a graph G = {V,E), let Vi = {vi \ v G V} and Vr = {vr \ v G 
V} be two copies ofV, called the left and the right copies, respectively. Let D he a natural number 
less than re. The flow graph of G with threshold D, a source s and a sink t is a directed graph on 
nodes I 4 U W U {s, t} with the following capacitated edges: edges of capacity D from the source s to 
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all nodes in Vi and from all nodes in Vr to the sink t, and unit-capacity edges {u^, Vr) for all edges 
{u,v) of G. The flow graph of G is denoted FG(G). 

We would like our extension function to output the sorted list of flows leaving the source vertex 
in some maximum flow. The challenge is that there may be many maximum flows. If we select a 
maximum flow arbitrarily, then the selected flow may be very sensitive to changes in the graph, 
even though its value changes little. We get around this by selecting a flow that minimizes a strictly 
convex function of the flow values. 

Definition 4.2 (Lipschitz extension of degree list). Given a flow f of FG(G), let /(e) denote the 
flow on an edge e. Also, let fg, be the vector of flows on the edges leaving the source s, let f,t be 
the vector of flows on the edges entering t, and let fs»,»t be the concatenation of the two vectors. 
We use D 2 n to denote a vector of length 2n, where all entries are D. Let ^{f) be the squared £2 
distance between fs»,»t o,nd D 2 n, that is, 

Hf) = -D2n\\l = Yl ((^ - ^))') • 

v£V 

Let f be the flow that minimizes the objective function <I> over all feasible flows in FG(G). Define 
fniG) to be the sorted list of flows along the edges leaving the source, that is, foiG) = sort{f,t)- 

The function fniG) is uniquely defined because the objective <I> is strictly convex in the val¬ 
ues fs»,mt- foiG) can be approximated to arbitrary precision in polynomial time, since it is the 
minimum of a strongly convex function over a polytope with polynomially many constraints. The 
approximation may slightly increase the sensitivity; in our application, one can acconnt for this by 
adding slightly more than 3Dfe noise in each coordinate. 

Theorem |1.3| follows from the following theorem. 

Theorem 4.3. The function fniG) is a Lipschitz extension of deg-list{G) from to Q of 
stretch 3/2. In other words, 

1. If G is D-bounded, then fniG) = deg-list{G). 

2. For any two graphs Gi,G 2 (not necessarily D-bounded) that are node neighbors, 

\\fD{Gi)-fD{G2)\\i<3D. 


Proof of Theorem j.S (item 1). The flow that assigns 1 to all edges {ui,Vr) and deg{v) to all edges 
{s,v/j and {vr,t) strictly dominates all feasible flows. In particular, it minimizes since, for 
X G [0, D], function [D — x/ is decreasing in x. □ 


There are two distinct notions of optimality of a flow in FG(G): optimality with respect to $, 
which we call ^-optimality, and optimality of the net flow form s to t, called net flow optimality. 
Next, we show that $-optimality implies net flow optimality. 


Lemma 4.4. For every graph G, if f minimizes <1> among valid flows for the flow graph FG(G), 
then f has maximum net flow from s to t in FG(G). 
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Proof. If / does not have maximum net flow, then we can find a shortest augmenting path p from 
s to t. Let c > 0 be the minimal residual capacity of the edges in p. Since p is a shortest path, it 
is simple; thus, adding cp to / results in a feasible flow, but does not decrease the flow along any 
edge leaving s or entering t. This implies that <h(/ + cp) < ‘h(/) (since is strictly decreasing in 
each argument), contradicting the <I>-optimality of /. □ 


The flow graph FG(G) admits a simple symmetry: for any flow /, we can obtain a feasible flow 
7r(/) by swapping the roles of s and t and the roles of left and right copies of all vertices. That is, 
we define 7r(/)(s,U£) := f{vr,t), TT{f){ur,t) := f{s,Ui), Tr{f){ui,Vr) := f{ve,Ur) for all vertices u, u 
in G. Flow / is symmetric if 7r(/) = /. For every graph G, there exists a symmetric <h-optimal flow 
in FG(G): given any <I>-optimal flow /', the flow f" = ^{f' + 7r(/')) is symmetric, feasible (because 
the set of feasible flows is convex) and has objective value at most <h(/') by convexity of 


Proof of Theorem 4-3 (item 2). Suppose a graph Gi on n vertices is obtained by removing a node 
along with its associated edges from a graph G 2 (on n + 1 vertices). 

Let /i ,/2 be <h-optimal symmetric flows for the flow graphs FG(Gi) and FG(G 2 ), respectively. 

Observe that fi is a feasible flow in FG(G 2 ). Consider the flow A = f 2 — fi- Note that A is 
a maximum signed flow in the residual graph of flow fi for FG(G 2 ). In particular, A satisfies flow 
and capacity constraints, but not necessarily positivity. Since ||/d(Gi) — /d(G 2 )||i = ||As,||i, our 
goal is to prove ||As,||i < 3D. 

Next, we decompose A into three subflows. A subflow of a flow A is a flow A' such that for 
all edges e, the flows A(e) and A'(e) cannot have different signs and |A' (e)|< A(e). We start by 
decomposing A into subflows that form simple s-t paths and simple cycles. Then we group them 
as follows: 


• Let A® be the sum of all flows from the initial decomposition that form paths and cycles using 
the edge (s,u"®’"). 

• Let A* be the sum of all flows from the initial decomposition that form paths and cycles using 
the edge (u“",t), but not (s,u"®"). 

• Let A*^ be the sum of the remaining flows, i.e., A^ = A — A'^ — Ah 

Since, by definition of the subflow decomposition, ||As,||i = ||A®s,||i + ||A*s,||i + ||A°s,||i, it remains 
to bound the three values in the sum. We do it in the following three lemmas. 

Lemma 4.5. ||A®s,||i < 2D. 

Proof. Recall that A® can be decomposed into simple s-t paths and simple cycles that use the 
edge (s,?;”®"). Each such path contributes the value of its flow to ||A®s,||i, and each such cycle 
contributes at most twice the value of its flow. Since the total flow A^(s,u“") is at most D, we 
get that ||A^s,||i < 2D. □ 

Lemma 4.6. ||A*s,||i < D. 

Proof. Recall that A^ can be decomposed into simple s-t paths and cycles that use the edge (u)!®", t), 
but not (s,u“®'^). Each such path contributes the value of its flow to ||A^ 5 ,||i. Any such cycle 
contributes 0 to ||A'^s,||i because any simple cycle in A that starts from t cannot reach s. If it did, 
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one could find an angmenting s-t path in A, implying that /2 is not a net value optimal flow in 
FG(G 2 ) and, by Lemma [44| contradicting <h-optimality of /2 in FG(G 2 ). 

Since the total flow is at most D, we get that ||A* 5 ,||i < D. □ 

Lemma 4.7. ||A°s,||i = 0. 

Proof. The flow A° does not use the edges and since all flow in A along 

and has been used by A* + A*. Consequently, A*^ has no flow passing through and 

^new^ Therefore, A^ is a feasible flow for the residual graph of /i in FG(Gi). We conclude that 
/i + A° is feasible in FG(Gi). 

Suppose for the sake of contradiction that ||A*^s,||i > 0. Then we can use convexity of <f> to 
prove the following inequalities: 

(A0,D2„-/i) < 0. (2) 

{A°,D2n-(f2-A^)} > 0. (3) 


To prove Q, consider the polytope of feasible flows in FG(Gi). Both fi and fi + A^ are in the 
polytope. Moreover, fi is the unique <l>-optimal flow in FG(Gi). Since is minimized at L> 2 n, a 
tiny step from fi in the direction of fi + A*^ takes us further from D 2 n- In other words, the angle 
between the vectors (/i, D 2 n) and (/i, /i + A°) is at least 90°, implying ([^. 

To prove Q, consider the polytope of feasible flows in FG(G 2 ). Both /2 and /2 — A° are in 
that polytope. Moreover, /2 is the unique <h-optimal flow in FG(G 2 ). Since $ is minimized at D 2 n, 
a tiny step from /2 — A*^ in the direction of /2 takes us closer to D 2 n- In other words, the angle 
between the vectors (/2 — A°,/ 2 ) and /2 — A^,D 2 n) is less than 90°, implying O. 

Subtracting ([^ from Q and using the fact that A = /2 — /i = A^ + A* + A°, we get 

(A0,D2n-(/2-A0))-(A°,.D2n-/i) > 0; 

(A°,-(/2-/i-A0)) > 0; 

(A°,A" + A*) < 0. (4) 


But A*^ and A^ + A* are both snbflows of A, so they cannot have opposite signs, on any edge, 
contradicting Q. Therefore, ||A‘’s,||i = 0. □ 


A*, 

IIA" 


We now complete the proof of Theorem 4.3 ( Item 2). Recall that A = A^ + A* + A*^ and that 
A*, and A*^ are subflows of A. From Lemmas 


|i + l|A*s»||i + ||A*^s,||i < 3D, as desired. 


4.5 


4.7 


we get ||/d(G'i) -/d(G 2 )||i = ||A^,||i = 


1 = 

□ 


4.2 From the Degree List to the Degree Distribution 

Let pg denote the degree distribution of the graph G, i.e., pcik) = lln : deg„(G) = A:}|/|F|. 
Similarly, Pq denotes the cumulative degree distribution (CDF), i.e., Pcik) = |{n : deg.y(G) > 
k}\/\V\. 

We can modify the extension of the degree list to get extensions of the degree histogram n ■ pc 
or the cumulative degree histogram (CDH) n ■ Pq. If we consider two integral degree lists that 
are at ii distance t, then the ii distance between their CDH’s is at most t (similarly for degree 
histograms). However, since our extension of the degree list may produce fractional lists, we need 
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to extend the CDH to fractional degree lists so that the map from lists to CDHs remains Lipschitz 
in the norm. 

We do this first for the CDH; the extension of the degree histogram is an easy modification. 
Given an integer k G [D], let 


[x]k = max{ 0 , min{l, x — {k — 1 )}} 


0 \i X < k — 1, 

' X — {k — 1) if A; — 1 < X < /c, 

1 \i x> k. 

\ — 


Define the map H as follows: for a nonnegative real number a, H{a) = ([a]i, [a] 2 ,..., [a][a])- (This 
is a vector of length \a\ whose norm is exactly a.) Given a finite sequence (oi, ...,an) G [0,D]*, 
let H{ai, .■■,an) = where we pad shorter sequence with O’s to allow summation. If the 

input numbers are in [0, D], the sequence has length at most D. 

Lemma 4.8. The function H is 1-Lipschitz in the ii norm. That is, \\H(a) — H(a')\\i < ||a —a'||i 
for all vectors a, a' G [0, D]*. Moreover, for every graph G, H[deg-list{G)) = n- Pg where n = \ Vg\- 

Proof. This follows from the fact that H(a) has ii norm a for every nonnegative real number, and 
equals the sequence 1“ when a is an integer. □ 

Given H, which extends transforms degree lists to the CDH, we can obtain an extension of the 
degree histogram via hist£){a) = HD{a), and histi{a) = Hi{a) — for i < D. This increases 

ii distances by at most an additional factor of 2. 

Theorem 4.9. The map 

G ^ histifoiG)) 

extends the degree histogram (as a map from (^^,dnode) to i^) to Q, with stretch at most 3. 


4.3 Differentially Private Approximations to the Degree Distribntion 

There are two natural approaches to using the extension of deg-list to release an approximate 
degree distribution. First, we may add noise D/e to each entry of the sorted degree list, and 
project (and/or) remove noise as in [l2l HTJ El] • The second is to release the D-bounded degree 
histogram and add noise. The error of the first approach is difficult to bound analytically, and so 
we adopt the second here. 

Given a degree threshold D, consider the following mechanism: 


Algorithm 1: Noisy Degree Histogram(G, e, D) 

1 Tj ~ Lap( 6 D/e) for f = 1,..., D; 

2 return Ad{G) = hist{fD{G)) + (1/, ..., Yd); 


(We only need to release the first D entries of hist, since the remaining entries are always 0.) 
This mechanism introduces two sources of error: the extension error f{G) — deg-list(G) and the 
random noise Y = (Yi, ...,Yd). The noise component is easy to understand and bound. How can 
we characterize the error introduced by the extension? 
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Lemma 4.10. For any graph G and threshold D, the extension’s ii error satisfies 


n ^ Pcii) < ||/d(G') - deg-list{G)\\i < 2n ^ Pg(0 • 

i>D i>D 

Proof. Recall that f{G) has minimal £i error — deg-list(G)||i among all L>-bounded real 

vectors that are consistent with a weighted graph. In particular, one can consider a graph G’ 
which is obtained by removing deg„ —D edges for each vertex v with degree greater than D. Each 
edge removal causes an change of 2 in deg-list(G) in the ii norm. The number of edges removed 
Ylv deg ~^) ■ alternative formula for this sum can be obtained by summing over 

degrees instead of vertices: 

(deg^ -D) = Y nPcii) 

V. deg„>D i>D 

(since each vertex v contributes max(0,deg„ —D) to the sum). Multiplying by 2 yields the desired 
upper bound. 

To prove the lower bound, note that the vector foiG) is always less, coordinatewise, than the 
simple projection that replaces the degree deg^ of each vertex v by min(Zl, deg^). The ii error of 
f{G) (or indeed of any function that projects onto a set of vectors with entries bounded by D) is 
therefore at least J2v: deg„>D(deg^ -D) = n J2i>D Pcii)- □ 

Combining the two previous lemmas with the fact that the expected absolute value of each Yi 
is 4H/e, we obtain the following theorem. 


Theorem 4.11. The expected £i error of algorithm Ad on input G is at most 2n E Pg{^ + 


6T»2 


i>D 


This theorem bounds the error of the algorithm for a given degree threshold D. In the sequel, 
we show how we can select a (nearly) optimal threshold differentially privately. 

5 Exponential Mechanism For Scores With Varying Sensitivity 

The exponential mechanism of McSherry and Talwar m is a basic tool for designing differentially 
private algorithms. We present here a generalization for score functions with different sensitivities. 

Suppose the data set comes from a universe U equipped with an neighbor relation (e.g., Ham¬ 
ming or set-difference distance for standard data sets, or vertex distance on graphs). We assume 
that the set of possible answers is finite and index it by elements of [/c]. Given a collection of 
functions qi, .■.,qk from 17 to M and a private data set x G C/, the goal is to minimize qfix), that is, 
to find an index i such that qfix) ~ miuj qfix). Define 


Aj max \qi{x) - qi{x')\ 

x,x'£U adjacent 


and 


def 


^m,ax — max A.;. 


The exponential mechanism achieves the following accuracy guarantee: for every /3 > 0, with 
probability 1 — /3, the output i satisfies qfix) < min* qi{x) + Amax ■ ^ _ 

A limitation of this guarantee is that it depends on the maximum sensitivity of the score 
functions qi{-). In the context of threshold selection for graph algorithms, such a guarantee is 
meaningless for sparse graphs. This poor utility bound is not merely an artifact of the analysis. 
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Algorithm 2: Generalized Exponential Mechanism 
Input: Data set x from universe C/, parameters /3 G (0,1) and e > 0, 
score functions oi,oi. from U to M. 

1 Set t= 

2 for i = \ to k do 


3 


Aj = max 3 ;, 3 ;/g ;7 adjacent ki(a;) - /* An upper bound on Aj suffices. 

/* Generally, Aj is known exactly. 


4 for i = 1 to k do 

{qi{x) + tAi) - {qj{x) + tAj) 


s{i) 


max 

ie[fc] 


Aj + A,' 


/* s{i) has sensitivity at most 1 . 


6 return i •<— ExponentialMechanism{s{i), i 


*/ 

*/ 


*/ 


The problem is inherent in the algorithm. For example, consider the setting with k = 2, where 
the two score functions have sensitivity Ai = 1 and A 2 1. Further, consider a data set x with 
qi{x) = 0 and q 2 {x) = A^je. On input x, the exponential mechanism will select i = 2 with constant 
probability, resulting in an excess error of A 2 /e, which may be arbitrarily larger than Ai. 

In contrast, we give an algorithm whose excess error scales with the sensitivity of the optimal 
score function Aj*, where i* = argmin^ gj(x). Our mechanism requires as input an upper bound /3 
on the desired probability of a bad outcome; the algorithm’s error guarantee depends on this /3. 

Theorem |1.4| (Formal). For all parameters /3 G (0,1), e > 0, the generalized exponential mecha¬ 
nism (Algorithm^ is {e,0)-differentially private (with respect to the neighbor relation on U). For 
all inputs x, the output i satisfies 

qfix) < rnin (^qfix) + Aj • . ( 5 ) 


In particular, our algorithm is competitive with the sensitivity of the true minimizer i* = 
argminjgj(x) (since the right-hand side of ([^ is at most qi*{x) + Aj* • Jn the case that 

all the A's are the same, our algorithm simplifies to running the usual exponential mechanism with 
e' = e/ 2 ; this justifies the “generalized” name. 

The intuition behind the algorithm is as follows: since the score function q has different sen¬ 
sitivity for each i, we would like to find an alternative score function which is less sensitive. One 
simple score would be to compute, for each j, the distance, in the space of data sets, from the input 
x to the nearest data set y in which qj{y) is smallest among the values {qi{y)}i^[k] (this idea is 
inspired by the GWAS algorithms of |14) . This score has two major drawbacks: first, it is hard to 
compute in general; second, more subtly, it will tend to favor indices j with very high sensitivity 
(since they become optimal with relatively few changes to the data). 

Instead, we use a substitute measure which is both easy to compute (given the scores qi{x) for 
i G [A:]) and appropriately penalizes scores with large sensitivity. Given a value t > 0 (to be set 
later), define the normalized score as 


f ^ {Qi{x) + tAi) - {qffx) + tAj) 

sit; x) = max---- — = max 

ie[fc] Ai + Aj ie[fc] 


qfix) - qj{x) ^^Ai-Aj 


Ai -|- Ai 


Ai -j- A^ 


( 6 ) 
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The first term inside the maximum on the right-hand side is an approximation to the Hamming 
distance from x to the nearest data set y where score qj{-) becomes smaller than qi{-). The second 
term (containing t and independent of the data set) penalizes indices i with larger sensitivity. 

We obtain an index i by running the usual exponential mechanism on the normalized scores 
s{i). Our first lemma bounds the sensitivity of the normalized score. 


Lemma 5.1. For each i, and for any t G M, the normalized score s(z; •) has sensitivity at most 1. 

Proof. First, fix indices i,j G [k]. The ratio has sensitivity at most 1 since qi{-) and qj{-) 

can vary by at most Aj and Aj, respectively, when x changes to an adjacent data set. As long as 
t does not depend on x, the function s(i; •) is a maximum of sensitivity-1 functions, which means 
its sensitivity is at most 1. □ 


Proof of Theorem l.f. The algorithm sets t = 21n(/c//3)/e, regardless of the data x. Since the 


normalized scores have sensitivity at most 1, the application of the usual exponential mechanism 
(or its more efficient alternative, “report noisy min”) is (e, 0)-differentially private. 

To analyze utility, let i denote the index that minimizes the penalized score qi{x) + tAj. Then 


s(i;x) = 0, 


since each of the terms in the maximum defining s is nonpositive for i (and the term for j = i is 0). 
By the usual analysis of the exponential mechanism, we have that with probability at least 1 — /3, 


s(i; x) < s(i; x) + 
0 


21n(/c//3) 

e 


Now consider an arbitrary index j. Since s(i;x) is at least ^ multiply 

by Aj -|- Aj to obtain: 


qtix) < qj{x) + f (Aj - A;) + ■ (A; + Aj) 


Substituting t 


2in(k/y) desired result. 


□ 


5.1 Threshold Selection for Lipschitz Extensions 

Suppose we have a collection of candidate functions {/Ai}je[fc] for approximation a function /, each 
with sensitivity Aj. Moreover, the approximation functions are all underestimates, that is, 

/a,(G) < f{G) for all G and Aj. 

Let 

err(A,) = |/(G)-/A(G)| + A/6. 

This is a simple proxy for the (expected) error in approximating f{G) that one gets by using 
/a(G') + Lap(A/e). It exaggerates the expected error by a factor of at most 2, since the expected 
error is at most err(A) by the triangle inequality, and at least min(|/(G) — /a(G)|, A/e)). 
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The functions err(Aj) don’t necessarily have bounded sensitivity (since we make no assumption 
on how / varies). However, the differences err(Ai) — err(Aj) do have sensitivity at most A* + Aj, 
which allows us to employ the generalized exponential mechanism (alternatively, since the functions 
/Ai are all underestimates, we may use qi{x) = — /a(G') + ^/e)- 

Corollary 5.2. Running the generalized exponential mechanism with score qi{x) = err(Aj) and 
sensitivities Aj is differentially private and yields a threshold A such that, for every A* G {A*}, 
with probability at least I — 

err(A) < err(A*) + ^og{k/f ) ^ . q _ 

Selecting from a continuous interval of thresholds The extensions we consider satisfy a 
further guarantee of monotonicity, namely, if Ai < A2, we have 

/ai(G)</a,(G')</(G). (7) 

If we want to select among an interval [1, Amax] of possible thresholds, then this guarantee ensures 
that selecting among the powers of a fixed constant (e.g., 1, 2,4,..., g^yg 

multiplicative approximation to be best choice of A, since for all values of A > 0, 

err(A) < 2err(A/2). 

We obtain the following proposition. 

Proposition 5.3. If the collection of functions {/A}Ae[i,A,„aa:] forms a monotone family of approx¬ 
imations to f (as in 0;. then applying the generalized exponential mechanism to the powers of 2 
in the interval [1, A^ax] yields a threshold A such that, for every A* G [1, Amax]; with probability 
at least I — 

err(A) = err{A*) ■ O ^lnln(Amax) + 

This generalizes and improves on the result of Chen and Zhou [S], who gave a method for 
selecting a sensitivity threshold that was specific to their Lipschitz extensions and within a log(n) 
multiplicative factor (as opposed to log log n) of the optimal error. 

5.2 Selecting a Threshold for the Degree Distribution 

Consider the algorithm for releasing the degree distribution discussed in Section Recall that the 
algorithm’s error is at most 

err{D) ||/d(G) - deg-list(G)||i + 6T»^/e. (8) 

This error function is closely related to the error of the approximation to the number of edges 
from Kasiviswanathan et al. [20]. Specifically, let g{G) denote the number of edges in G, and go 
denote the Lipschitz extension of g from to Q. Then 

gD{G) = ||/d(G)||i and g{G) - goiG) = WMG) - deg-list(G)||i. 

We can therefore use the process above for selecting a threshold for a one-dimensional function 
to select a threshold for releasing the degree distribution. 
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Proposition 5.4. Given a graph G and parameter e, let D* = argmin£)g[]^^„] err(D). Applying 
the generalized exponential mechanism with qi{G) = err(2*), fori G {1, 2, 4,2l-*°§2(")J} is (e,0)- 
differentially private and yields a threshold D such that, for every A* G [1, Amax], with probability 
at least 1 — fd, 

E (^err{D)^ < 2err{D*) + = err{D*) ■ O ^lnln(n) +\n^ 

6 Error Analysis on o-Decaying Graphs 

Our techniques provide a significantly more accurate way to release the degree distributions of 
graphs while satisfying node-level differential privacy. To illustrate this, we study the accuracy of 
our method on graphs that satisfy a-decay, a mild condition on the tail of the degree distribution. 



6.1 a-Decay 

Recall that d{G) = 2|i?|/|l/| is the average degree of G. 

Assumption 6.1 (a-decay). Fix a > 1. 4 graph G satisfies a-decay if for a/0rea/ numbers t > 1, 
PG{t-d)<t-^. 


Note that all graphs satisfy 1-decay (by Markov’s inequality). The assumption is nontrivial 
for a > 1, but it is nevertheless satisfied by almost all widely studied classes of graphs. So-called 
“scale-free” networks (those that exhibit a heavy-tailed degree distribution) typically satisfy a- 
decay for a G (1,2). Random graphs satisfy a-decay for essentially arbitrarily large a since their 
degree distributions have tails that decay exponentially (more precisely, for any a we can find a 
constant Cq, such that, with high probability, a-decay holds when t > Ca). Regular graphs satisfy 
the assumption with a = oo. The following lemma bounds the number of edges adjacent to nodes 
with degree above a given threshold. 


Lemma 6.2. Consider a graph G on n nodes that satisfies a-decay for a > 1, and let D > d{G). 
Then 


E s 

v: deg„(G)>D 


d(GP 

- . 77 

(a-hl)T>“-i 


6.2 Error Analysis 

Kasiviswanathan et al. [20] gave algorithms for releasing the degree distribution using a projection- 
based technique. Their algorithm required knowledge of the decay parameter a (which was used 
to select the projection threshold). They bounded the ii error of their algorithm in estimating the 
degree distribution, and showed that it went to 0 as long as a > 2 and d was polylogarithmic in n. 
More precisely, they gave an expected error bound of 

IE Up — PgIIi = O . 

^Our results hold even when this condition is satisfied only for sufficiently large t. For simplicity, we use a stronger 
assumption in our presentation. 
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Combining the noisy Lipschitz extension of the degree histogram (Theorem 4.11) with the 
threshold selection algorithm (Proposition 5.4), we get an algorithm Acombo with much better 
accuracy guarantees that, additionally, does not need to know the parameter a. 


Algorithm 3: Degree Histogram Estimation For Unknown Threshold 

1 D Generalized Exponential Mechanism(q£)(-), D G {1, 2,4,...., }) using score 

gniG) = err{D) and sensitivity bound D ; 

2 h ^ + (ki,..., Yjy) where Yi ~ Lap(4i)/e) i.i.d. ; 

3 return j5 =/i/||/i||i; 


Theorem 6.3. Given inputs G £ Q and e > 0, the algorithm Acombo produces an estimate p such 
that, if G satisfies a deeay for a> 1, then 

IE Up — Pg111 = O ( (Inlnn)/ (en)“+i j . 

- 2a 

In particular, this error is o(l) as n goes to infinity if a > 1 and = o(en). 

Proof. Fix a graph G that satisfies a decay for a > 1, and let d denote its average degree. Con¬ 


ditioned on selecting a given degree threshold D, Theorem 4.11 guarantees that the ii error of 
our algorithm in estimating hist{deg-list{G)) = n • pc is at most err{D) = 2n Pg'(z) -|- 

(defined as in ([^). 

Although the true size of the graph n is not known to the algorithm it is convenient to divide 
everything by n so that we can compare to the true degree distribution pc. Let p = h/n denote the 
estimate of pc one gets by normalizing the estimated degree histogram by the true vertex count 
n rather than ||/i||i. We will account for the estimation of n at the end of the proof. Dividing 

by n, we get a bound of the form = Q (Yli>D the error in estimating pc- 

By Lemma 6.2 this bound is at most O (^dP/{{a -|- 1)D““^) -|- D^/(ne)). In particular, if we set 
^ (which makes the two terms in the sum equal) then the expected G error of p 

err{D*) 


D* = {d^en 
is at most 


n 


= 0 ((i“+i / (en) “+i). 


In the algorithm, we do not select D* but rather a differentially private alternative D. By the 
law of conditional expectations, the overall expected error of p is at most the expectation of 
that is, 


IE||p-Pg||i =IEf, (E- 


P-PG\\l 


d)) < (err(D)) 


By Proposition 


5.4 


the expectation of err{D) is at most 2err{D*) + 8D* lnln(n)/e. For a < oo, 
the reference threshold D* is polynomially large in n. Thus, the first term err{D*) (which is at 
least {D*fi/e) dominates the second term, and the final error bound is 

E Up — PgIIi = 0{d ^/ (en)^). 

Finally, we analyze the difference between Acombo and A'. Let h denote the estimated number 
of edges in G, that is n = ||/i||i. Note that for any given realization of the algorithm’s random 
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choices, if p is a good approximation to the true distribution pc, then n must be good estimate of 
the true number of vertices: 


|h — n| = I ||h||i — n I < n| ||p||i — 1 \ < n\\p — pcWi ■ 

This allows us to bound the difference between p and p. Since p — p = — l), the ii norm of 

p — p is at most ||/iatp||i||p — pclli = ||p — Pg||i- Thus, the error of p in estimating pc is never more 

than twice the error of p: 

— 2a Q —1 

Up “ PgIIi < 2||p — pgIIi and thus E\\p — Pg\\i = 0{d‘^+^ / {en)°‘+'^). □ 
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A Doubling dimension of graph metrics 

Definition A.l. The doubling dimension p of a metric space {X,d) is the smallest integer such 
that every ball in X can be covered using at most 2^ balls of half the radius. 

Consider the set Gn of graphs on at most n vertices. If we equip Qn with the edge adjacency 
metric, we get a set essentially equivalent to the ( 2 )-dimensional Hamming cube (in fact, a union 
of n different Hamming cubes corresponding to graphs of sizes 1, 2,..., n). This metric has doubling 
dimension 0 (n^). 

Intuitively, the doubling dimension of the vertex-adjacency metric on Gn should be similar. We 
sketch a weaker statement here, namely that the doubling dimension is 0(n). This bound shows 
that constructions with stretch bounded by the doubling dimension still have very high stretch 
when used on the vertex metric. 

Lemma A.2. The doubling dimension of the vertex-adjacency metric on Gn for D > 1 is Q{n). If 
we collapse the set Gn by identifying isomorphic graphs, then the statement continues to hold for 
D>A. 

Proof Sketch. Assume n is even, w.l.o.g. To prove the theorem, we embed the Hamming cube 
Hamn /2 into Gn ■> which shows that Gn has doubling dimension Yl{n). Let Go be a uniformly 
random regular graph of degree 3 on n/2 vertices. For every subset S C [|], let G 5 be the graph 
on ^ [S'] vertices obtained by starting from Go and adding one vertex for each element in S and 

connecting it to the corresponding vertex in Go- 


22 




The vertex distance between two such graph Gs and Gt is i7(|5AT|), as with the Hamming 
metric, as long as S and T are sufficiently far from each other. This is sufficient to prove the 
main result as we may select S and T from an error-correcting code with linear rate and minimum 
distance. 

A complete proof is delicate, since one must account for the possibility that one can get from 
Gs to a graph that is isomorphic to a subgraph of Gt by deleting fewer than |5AT| vertices from 
Gq. We omit the details. □ 
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